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Abstract. An intelligent emergency response virtual environment (ERVE) that provides emergency first 
responders, response planners, and managers with situational awareness as well as training and 
support for safe schools is presented. ERVE incorporates an intelligent agent facility for guiding and 
assisting the user in the context of the emergency response operations. Response information folders 
capture key information about the school. The system enables interactive 3D visualization of schools 
and academic campuses, including the terrain and the buildings’ exteriors and interiors in an easy to 
use Web-based interface. ERVE incorporates live camera and sensors feeds and can be integrated 
with other simulations such as chemical plume simulation. The system is integrated with a 
Geographical Information System (G1S) to enable situational awareness of emergency events and 
assessment of their effect on schools in a geographic area. ERVE can also be integrated with 
emergency text messaging notification systems. Using ERVE, it is now possible to address safe 
schools' emergency management needs with a scaleable, seamlessly integrated and fully interactive 
intelligent and visually compelling solution. 

INTRODUCTION find a wall where they expected to find an exit 

route. 

Emergency responders and planners often have 

to respond to man-made or natural emergencies In addition to the above issues, it is often the 

in buildings and facilities that they are not case that emergency responders need to be able 

familiar with. Examples of such facilities include to operate special equipment or emergency 

schools, ports, military facilities, commercial systems. For example, they may need to operate 

centers, government buildings, utilities an emergency generator or HVAC system that 

installations, bridges, tunnels, etc. they are not familiar with. 

The Columbine High School and Virginia Tech Also, there are typically many schools and 

experiences demonstrate the severity and school buildings in a geographic area. Therefore, 

magnitude of emergency incidents involving the system must be scaleable and must be able 

school populations. First responders and to easily and efficiently capture the knowledge 

emergency managers have a daunting task about those facilities. Moreover, because of the 

when it comes to incidents in school facilities. dynamic nature of school facilities, the system 

When an emergency occurs, emergency must enable easy renewal of this knowledge, 

responders and planners need to quickly locate 

special areas of concern in the school. Currently, In this paper, a web-based Emergency 

they often have to spend a lot of time getting and Responders Virtual Environment (ERVE) that 

studying architectural drawings or browsing addresses the aforementioned issues is 

through large volumes of information about the presented. The application of ERVE to model a 

layout of the school and the systems and high school facility is presented in order to 

functions of each area in the school. This can be demonstrate the features of the system, 

a time consuming task, especially if the 

information is not easily accessible or is in paper SURVEY OF VIRTUAL SPACE MODELING 
form. It is often the case that such information AND SIMULATION SYSTEMS 
has to be procured manually and this can take 

valuable time in emergency situations where This brief survey will be focused on systems that 

every second counts. Moreover, even when this are particularly suitable for modeling interaction 

information is procured, it often doesn't reflect within virtual facilities and that can be used for 

the actual layout of the school building as recent training as well as for supporting emergency 

upgrades and modifications may be done without response planning and operations. For example, 

updating the architectural plans. Thus, for systems such as those developed specifically for 

example, emergency responders may actually architectural visualization have not been 
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included. Such systems provide very high quality 
visualization "movies," typically showing a 
prerecorded flythrough or walkthrough 
visualization of a facility but provide little or no 
interactivity with the virtual space. 

The significant increase in processing power 
available for rendering in recent years has led to 
the development of many virtual space 
simulators for providing users with operational 
support and training in a "near-natural" synthetic 
environment. Applications of the virtual space 
simulators include training for operation of 
industrial machines [1,2], power-plants [3], 
vehicle driving, piloting, traffic-control, 
maintenance simulators [4], medical procedures 
training [5,6], emergency response training [7], 
and military operations training [8]. 

Misra et al. [7] developed an application called 
FiRSTE that attempts to address the training 
needs of first response personnel. The project 
aim was to determine how effective virtual 
environments can be for training first responders 
to meet the difficult challenges of responding to a 
chemical attack or hazardous material (Hazmat) 
incident. The use of virtual reality technology in 
this application can potentially provide training 
that is safe, flexible, and cost-effective. The 
system attempts to train first responders in 
applying proper procedures, it is designed to 
accommodate personal protection equipment 
and to be stressful and challenging. The FiRSTE 
system was built using the commercial Half- 
Life® game engine. Some of the issues that are 
still being worked on include addressing issues 
of hardware and software reliability and detailed 
functionality. 

Forterra Systems and Stanford University 
Medical Media and Information Technologies 
Center (SUMMIT) collaborated on research to 
apply game technology to the training of medical 
emergency first responders [9]. The research 
focuses on pre-hospital and in-hospital medical 
first response to WMD events simulated in a 
virtual training environment. The system relies on 
human actors controlling avatars in the virtual 
environment using a multi-player game 
paradigm. Hence, the effectiveness of the 
system will depend on finding suitable human 
actors and how well they "act” their part. 

McGrath et al. [10] used game-based simulation 
to develop two emergency response 
applications, the Unreal Triage and the Unreal 
Tunnel. They developed those simulations using 
the Unreal Tournament (UT) game engine. In the 
Unreal Triage simulation, the first responder 


arrives at the scene of a mass casualty event. A 
“heads up” display provides information on vital 
signs and injuries of the various victims [11], The 
Unreal Tunnel provides a simulated scenario of 
the detection and inspection of a suspicious 
vehicle at a tunnel toll plaza. 

Systems that rely on game engines are 
inherently constrained by the limitations of the 
game engine. In general, game environment 
creation software requires extensive effort by a 
large number of game developers to give the 
end product professional polish. Setup can be 
frustrating and technical support lacking. There 
are still very few industry standards regarding 
features and user interface. Data exchange 
between game applications and industry- 
standard CAD software is spotty at best. Popular 
game engines can run on Microsoft Windows but 
not Unix/Linux, Mac, or portable devices and are 
non-web based so they would not be easily 
accessible or deployable. Some games tend to 
be hard to use, especially for novice computer 
users; thus, tutoring and guidance often must be 
done by a human instructor prior to using the 
game system. 

There is a need for emergency responders 
training and support system that realistically 
represents school facilities and that can be 
efficiently built and be easy to deploy. By having 
access to a highly realistic and interactive virtual 
model, emergency responders can become 
familiar with the school building and know what 
to expect in an emergency situation. The system 
must be easy to use and must enable 
emergency responders to rapidly gain 
knowledge about the school, locate key areas 
and systems, provide instructions on how to 
operate key systems, and point out locations of 
potential problems. 

The Emergency Response Virtual Environment 
(ERVE) addresses the aforementioned needs by 
seamlessly integrating the following features: 

(1) An intelligent agent facility. 

(2) Response information folders that provide 
key information about the facility and 3D 
models and 2D floor plans as well as 
navigable panoramic digital images. 

(3) Live camera and remote sensors feeds. 

(4) Geographical Information Systems (GIS). 

(5) Chemical plume simulation. 

(6) Emergency Text Messaging Notification. 

INTELLIGENT AGENT FACILITY 
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The ERVE intelligent agent [12] provides natural- 
language instruction with naturally sounding text- 
to-speech and synchronized multimedia. A near 
photorealistic animated humanoid provides 
natural interaction with the user. The intelligent 
agent can provide debriefing about the school 
and emergency events as well as instructions on 
key elements or areas of the school or campus. 
The agent can provide instructions on how to 
operate critical systems such as the emergency 
generator, HVAC system or the Fire Alarm 
Control Panel. For example, in figure 1 the agent 
provides information about a chemical HAZMAT 
incident event. 



Figure 1: Agent providing event debriefing 


As seen in figure 2, in the high school facility 
application, the agent is providing a short 
multimedia debriefing that can be given to 
emergency responders to quickly and effectively 
familiarize them with the school that is impacted 
by the emergency event. 


Figures 3 and 4 show the intelligent agent 
pointing out the location of the Fire Alarm Control 


Panel (FACP) and then, if needed, providing 
detailed instructions on how to operate it. 



Figure 3: ERVE pointing out the location of 
FACP 



operations 


The key elements of the intelligent agent are: (1) 
a modular knowledge-base in which knowledge 
is stored with embedded synchronization with 
multimedia content; (2) Natural-language text-to- 
speech capability; (3) animated human-like 
virtual humanoids; (4) an integrated web-based 
framework. 

Modular knowledge-base 

The knowledge base contains knowledge items. 
The knowledge items have information about a 
particular element of the school and the related 
multimedia as well as intelligent agent gestures 
and emotions associated with the specific 
knowledge item. The knowledge is in XML format 
and thus can be updated using XML-enabled 
synchronous or asynchronous messaging 
services. Knowledge is disseminated by passing 
the text of the knowledge item to the speech 
synthesis engine to produce the natural- 



Figure 2: Intelligent agent providing debriefing 
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language speech and the multimedia is 
automatically synchronized with its 
corresponding knowledge item. In addition, any 
gestures and/or emotions scripts that are 
associated with the knowledge item are passed 
to the virtual humanoid and these scripts 
animate the humanoid accordingly. 

Natural-language text-to-speech 

Most web-based training or presentation systems 
are based on the user reading the material and 
looking at static or animated illustrations. Some 
systems present the materia! using pre-recorded 
speech with Flash animations and movies. 
Prerecorded speech makes the content very 
hard to maintain and update because any small 
change in wording involves spending a 
significant amount of time re-recording speech 
(as well as having to use the same person that 
recorded it the first time) and reprogramming the 
multimedia synchronization. In addition, 
synchronizing the speech with the Flash content 
using pre-recorded speech is very hard (the 
speech must be broken down into short phrases) 
because of the variation in Flash playing speed 
under different conditions of user interactions. 

The ERVE intelligent agent facility uses a 
speech synthesis engine to provide natural- 
language text-to-speech. Speech synthesis is 
done using any SAP! 5.1 compatible text-to- 
speech voice library to synthesize the agent 
voice. The use of text-to-speech rather than pre- 
recorded speech makes updating the content of 
the speech extremely efficient as it is simply a 
matter of typing the new information in the 
knowledge base. Moreover, the multimedia 
animation is automatically resynchronized with 
the speech at any speech or animation speed 
and no additional effort is required to re- 
synchronize the animations. 

Virtual humanoids 

The ERVE intelligent agent is visually 
represented in the virtual environment using 
photorealistic animated virtual humanoids. The 
appearance of the humanoids can be 
customized based on the application. To relate 
to emergency responders, we created policeman 
and fireman humanoids to disseminate the 
knowledge to the users. The humanoids are able 
to gesture and display emotions. The virtual 
humanoids obtain the visemes from the speech 
synthesis engine to provide automatic lip- 
synching. They also obtain the gestures and 
emotions script and display them accordingly. 
The photorealistic animated virtual humanoids 


present the knowledge about the school with 
synchronized gestures and emotions and lip- 
synching and enable the intelligent agent 
interaction with the user to be extremely natural 
and compelling. 

Integrated web-based framework 

ERVE is fully web-enabled so it can be easily 
added to any web page. The web-based 
framework seamlessly integrates the ERVE 
intelligent agent facility with multimedia 
presentations, natural-language text-to-speech, 
photo-realistic animated humanoids, 3D models 
and 2D graphical content. 

RESPONSE INFORMATION FOLDERS 

Key information about individual critical facilities 
is collected and organized in a Response 
Information Folder System (RIFS). The folder 
allows emergency responders, managers, and 
planners to quickly get the information they need 
about the specific school. The folder is organized 
in 25 standard categories. Figure 5 shows the 
information folder for the high school application. 



Figure 5: Response information folder 


3D Models and 2D Floor Plans of the Facility 

RIFS integrates critical information about the 
school with 2D and 3D visualization of the 
building interiors as well as the terrain around 
the school. This allows the user to interactively 
explore the facility and get a highly realistic and 
accurate view. For example, the user can 
navigate to a specific area of the school, such as 
the location of a critical system. The intelligent 
agent is integrated with RIFS so that it can point 
the attention of the user to the system and, if the 
user requires, offer to provide instructions on 
how to operate it (Figure 3). 


534 


Navigable Panoramic Digital images 

RiFS use of interactive panoramic digital images 
greatly reduces the effort required to model the 
school since it makes it unnecessary to build 3D 
models of rooms' interiors, furniture, etc. 
Specialized digital photography equipment and 
software that process and produce the 
panoramic images allows each room to be 
captured in one shot that can be quickly set up 
and processed. 



Figure 6: Floor plan visualization with 
panoramas 


The high fidelity of the panoramas enables 
emergency responders to identify intricate details 
down to what type of flooring or doors they are 
going to encounter and to prepare for potential 
problems or obstacles. For example, the 
panorama in Figure 6 shows the school 
cafeteria. The paper plans showed openings 
where there are steel doors at the end of the 
cafeteria. So contrary to what the paper plans 
indicate, this would have been a difficult 
evacuation route. 

LIVE CAMERA AND SENSORS FEEDS 

The system provides the capability to display live 
feeds from IP cameras and remote sensors. This 
allows the user to get, for example, valuable real- 
time video of the actual situation developing in 
the school or around its perimeter. 

GEOGRAPHICAL INFORMATION SYSTEMS 
(GIS) 

Geographic Information Systems (GIS) allow 
capturing, storing, managing, visualizing, and 
analyzing data and associated attributes which 
are spatially referenced to the earth. In a more 
generic sense, GIS is a tool that allows users to 
perform queries, analyze the spatial information, 


and edit and visualize geospatial data. ERVE 
can be seamlessly integrated with GIS. Thus, in 
the GIS, the user can manage various critical 
sites in a geographic area and associated 
geospatial information and visualizations. For 
example, as shown in Figure 7, the user can use 
the GiS to display all the schools in a district and 
overlay the plume produced by a simulated or 
actual chemical accident or attack based on the 
output of a simulation model or visualization of 
real-time sensors data. The user can then locate 
the critical facilities that would potentially be 
affected and interface to ERVE to get 
comprehensive information about those facilities. 



Figure 7: GIS Integration 


CHEMICAL PLUME SIMULATION 

ERVE is integrated with ALOHA, which is an 
atmospheric dispersion model. It is the primary 
software used by emergency responders for 
simulating releases of hazardous chemical 
vapors. Figure 8 shows a program developed in 
Java, which provides the integration with 
ALOHA. 



Figure 8: ALOHA Integration 
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CONCLUDING REMARKS 



Figure 9: GIS and plume modeling integration 


Figure 9 shows the visualization of the ALOHA 
output image of a chemical plume simulation of a 
mock chemical Hazmat incident in a semi- 
transparent overlay in the geographic area of the 
incident. Seeing that this event affects a 
particular school, the user can click on the school 
pushpin and open its associated Response 
Information Folder as in figure 5 to quickly pull 
detailed information about the school. 

EMERGENCY TEXT MESSAGING 
NOTIFICATION 

In an era where the use of cell phones is 
ubiquitous, text messaging is one of the effective 
ways to reach as many members of a target 
audience as possible in a short period of time. 
ERVE can be integrated with commercial text 
messaging broadcast systems to notify the 
school population of emergencies and 
appropriate actions. For example, a text 
message can be sent instantaneously to staff 
and students that classes are cancelled and that 
the school is closed due to an emergency. 



Figure 10: Example of ERVE integration with 
commercial text messaging systems 


In the present paper, a web-based Emergency 
Responders Virtual Environment (ERVE) was 
presented. The system can be used to train and 
familiarize emergency responders with school 
facilities as well as for support in actual 
emergencies. The system is integrated with GIS 
to allow users to manage all the school facilities 
in a geographic area as well as monitor 
emergency events and assess their impact on 
school facilities. The system guides the 
emergency responders to critical areas in the 
facility such as locations of hazardous chemicals, 
water and gas shut-off valves, fire suppressant 
equipment, and electrical panels. It provides 3D 
and 2D visualization of building layout, floor 
plans, exterior, aerial and surrounding structures. 
Human-like interactive and emoting intelligent 
agents provide debriefing and guidance to the 
user as well as instructions on how to perform 
complex tasks such as starting up a generator or 
the use of the fire control panel. ERVE can also 
be seamlessly integrated with emergency text 
messaging notification systems. The ERVE 
solution provides renewable on-demand facility 
knowledge and can be deployed across PC or 
portable devices via the Web. The seamless 
integration of modeling and simulation, virtual 
reality, GIS, remote sensing technologies, text 
messaging and intelligent agent technologies 
have the potential to dramatically improve how 
emergency responders and planners prepare for 
and respond to school emergencies. The 
application of ERVE to an actual high school 
facility was presented in order to demonstrate 
the features of the system. 
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VIII. Student Competition Session 

An Overview of Student Modeling and Simulation Competitions for 
Transportation System Design 
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